This paper theoretically explains the intuition that simple concepts are more likely to be learned by deep neural networks (DNNs) than complex concepts. In fact, recent studies have observed [24, 15] and proved [26] the emergence of interactive concepts in a DNN, i.e., it is proven that a DNN usually only encodes a small number of interactive concepts, and can be considered to use their interaction effects to compute inference scores. Each interactive concept is encoded by the DNN to represent the collaboration between a set of input variables. Therefore, in this study, we aim to theoretically explain that interactive concepts involving more input variables (i.e., more complex concepts) are more difficult to learn. Our finding clarifies the exact conceptual complexity that boosts the learning difficulty.

This paper theoretically explains the intuition that simple concepts are more likely to be learned by deep neural networks (DNNs) than complex concepts. In fact, recent studies have observed [24, 15] and proved [26] the emergence of interactive concepts in a DNN, i.e., it is proven that a DNN usually only encodes a small number of interactive concepts, and can be considered to use their interaction effects to compute inference scores. Each interactive concept is encoded by the DNN to represent the collaboration between a set of input variables. Therefore, in this study, we aim to theoretically explain that interactive concepts involving more input variables (i.e., more complex concepts) are more difficult to learn. Our finding clarifies the exact conceptual complexity that boosts the learning difficulty.

Learn how to use plain Python to create neural networks. Understand how Softmax, ReLU and Sigmoid allow you to approximate complex non-linear prediction functions. Realise that neural networks are not magic and can be implemented without using libraries, in any language you desire. Learn how to use plain Python to create neural networks. Understand how Softmax, ReLU and Sigmoid allow you to approximate complex non-linear prediction functions. Realise that neural networks are not magic and can be implemented without using libraries, in any language you desire.

  Udemy Coupon ED | Machine Learning for beginners with project Become a DeepLearning Pro with these valuable skills. Start Your Cours...

We're team of Machine Learning experts, AI developers working together to advance the state of the art in artificial intelligence. You will be hearing from us when new courses are released, answering Q&A and many more. We are here to help you stay on the cutting edge of Data Science and Technology.

Instead of relying on a list of rules or training on a massive data set like standard computers, a new computational framework for learning lets robots come up with their own concepts by detecting abstract differences in images and then recreating them in real life. Watch the video to learn more.